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Executive Summary 


Introduction 


The Multiple Measures Assessment Project (MMAP)? is a collaborative effort led by the 
Research and Planning Group for California Community Colleges (RP Group) and the 
Educational Results Partnerships’ (ERP) Cal-PASS Plus system, with input and support from the 
California Community College Chancellor’s Office (CCCCO) and the Academic Senate for 
California Community Colleges (ASCCC). MMAP seeks to develop, pilot, and assess 
implementation of enhanced multiple measures, including the use of high school performance 
and non-cognitive variables, in student placement and advising. This project was part of the 
California Community College’s Common Assessment Initiative (CAI) and now is supporting the 
implementation of AB 705,? which requires “that a community college district or college 
maximize the probability that a student will enter and complete transfer-level coursework in 
English and math within a one-year timeframe and use, in the placement of students into 
English and math courses, one or more of the following: high school coursework, high school 
grades, and high school grade point average.” 


The MMAP research team has conducted this study in order to examine whether student 
completion of a transfer-level course in math or English within one year varies by students’ 
participation in either the Extended Opportunities Programs and Services (EOPS) program or 
the Disabled Students Programs and Services (DSPS) program. In addition, this study seeks to 
examine if throughput rates vary in relation to whether a student enters directly into a transfer- 
level English or math course compared to beginning the sequence one level below transfer. 


The data examined in this report are disaggregated by students’ EOPS and DSPS status and 
across high school grade point average (HSGPA) ranges associated with different levels of 
college course performance as determined by the Multiple Measures Assessment Project 
decision trees for transfer-level English, statistics, and pre-calculus. Our research revealed that 
while variations occur within groups, students placed directly into transfer-level courses 
completed those courses at a higher rate than those placed one level below. 


1 http://rpgroup.org/All-Projects/ArticleView/articleld/118/Multiple-Measures-Assessment-Project- MMAP 


* https://assessment.cccco.edu/ 
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Key Findings 
EOPS 


The analysis conducted through this project produced the following key findings: 


e OPS students across all ranges of high school performance are one to two times more 
likely to complete transfer-level English if they start at transfer level than if they start one 
level below transfer. 


e OPS students across all ranges of high school performance are two to five times more 
likely to complete a statistics course if they start at that level than if they start one level 
below any transfer-level math course. 


e OPS students across all ranges of high school performance improved their throughput by 
almost 30 percentage points when placed directly into pre-calculus as opposed to starting 
one level below. 


DSPS 


e DSPS students across all ranges of high school performance are one to three times more 
likely to complete transfer-level English if they start at transfer-level than if they start one 
level below transfer. 


e DSPS students exhibited throughput rates more than two times higher than non DSPS 
students when placed into statistics than when placed below any transfer-level math 
course. 


Conclusion 


This analysis did not find evidence that students would have higher throughput rates by being 
placed into basic skills courses, whether based on students’ EOPS or DSPS status or high school 
performance. Thus, within the timeframe of data availability and given the curricular design and 
support structures that existed systemwide at this time, we are unable to identify any group of 
students who complete transfer-level English, statistics, or pre-calculus courses at a lower 
rate when placed directly into those classes rather than one level below transfer. 
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Introduction 


Project Background 


The Multiple Measures Assessment Project (MMAP)? is a collaborative effort led by the 
Research and Planning Group for California Community Colleges (RP Group) and the 
Educational Results Partnerships’ (ERP) Cal-PASS Plus system, with input and support from the 
California Community College Chancellor’s Office (CCCCO) and the Academic Senate for 
California Community Colleges (ASCCC). MMAP seeks to develop, pilot, and assess 
implementation of enhanced multiple measures, including the use of high school performance 
and non-cognitive variables, in student placement and advising. This project was part of the 
California Community College’s Common Assessment Initiative (CAI) and now is supporting the 
implementation of AB 705,4 which requires “that a community college district or college 
maximize the probability that a student will enter and complete transfer-level coursework in 
English and math within a one-year timeframe and use, in the placement of students into 
English and math courses, one or more of the following: high school coursework, high school 
grades, and high school grade point average.” 


Purpose of the Project 


There are two objectives of this report. The first objective is to examine whether student 
completion of a transfer-level course in math or English within one year (two semesters or 
three quarters) varies by participation in Extended Opportunities Programs and Services (EOPS) 
or Disabled Students Programs and Services (DSPS). The second objective is to determine any 
variance in throughput rates based on whether a student’s first course in the discipline is at 
transfer-level or one level below transfer. Both of these data analyses were disaggregated 
across high school grade point average (HSGPA) ranges as determined by the Multiple 
Measures Assessment Project (MMAP) decision trees for transfer-level English, statistics, and 
pre-calculus, and by EOPS and DSPS status. It should be noted that within the data file available 
for analysis, only indicators for students’ participation in EOPS and DSPS were available. 


For EOPS and DSPS status, success rates in the gateway transfer-level course for students who 
started at transfer level? were compared to the successful completion of a gateway transfer- 
level course within one year for students who began one level below transfer for transfer-level 
English, statistics, and pre-calculus. Statistics was chosen as a key example of a transfer-level 
math course for students pursuing a social sciences or liberal arts pathway, and pre-calculus 


3 http://rpgroup.org/All-Projects/ArticleView/articleld/118/Multiple-Measures-Assessment-Project-MMAP 
* https://assessment.cccco.edu/ 


5 Throughout rates for students who start at transfer-level or one term success rates. 
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was chosen as the most rigorous transfer-level entry point to a STEM pathway occurring after 
intermediate algebra across the California Community Colleges (CCCs).° 


In this Report 


This report presents the throughput rates for students whose first course at a California 
Community College was either at transfer-level or one level below transfer in English, statistics, 
or pre-calculus to determine whether observations of higher throughput for direct placement 
are consistent across subpopulations in the CCCs. In the first section of the report, we briefly 
provide an important context for this study by describing MMAP’s use of decision trees and the 
ways in which MMAP analyses are now being utilized to advance compliance with newly 
adopted AB 705 requirements. The remainder of the report presents the success rates of 
students who entered English or math at transfer level and also analyzes the throughput rates 
for students who began at one level below and then went on to enroll in the transfer-level 
course. These success and throughput rates are examined by students’ EOPS and DSPS 
participation and are further disaggregated by students’ level of high school performance. The 
report concludes with a summary of the primary takeaways and overall recommendations. 


Methodology 


In order to make best use of the data analyses described in this report, it is important to be able 
to place this project in the context of the overall MMAP initiative. This section describes the 
core work of MMAP and articulates how that work is now being focused on enabling 
compliance with the recently adopted Assembly Bill (AB) 705. 


The MMAP decision tree’ analyses include all students enrolled in an English or math course in 
the CCC system who also had four full years of high school data available in the California 
Partnership for Achieving Student Success (Cal-PASS Plus) data system, resulting in about 
250,000 cases. As such, the MMAP research is based on retrospective data—students who had 
already been placed, enrolled, and completed courses in CCCs, mostly between 2007 and 2014. 
The decision trees summarize students’ success rates using course grades in the first English or 
math course they completed at a CCC and information on students’ high school achievement. 
Example high school achievement information included in the models includes cumulative 
unweighted high school grade point average (HSGPA), highest courses completed, type of 
course completed (by level and subject matter), and time between high school and college. 


6 Starting in pre-calculus represents the highest possible bar and students with the same level of performance in 
high school would be expected to be less successful if placed directly there than if placed in a lower starting course 
like college algebra or trigonometry. Thus, if students placed directly into pre-calculus are more likely to 
successfully complete a transfer-level math course if placed there directly, similar students placed into college 
algebra, for example, would be even more likely to successfully complete that course when placed there directly. 


7 Decision trees are a form of data modeling that results in a set of “if-then” rules, where the if-then statements 
are referred to as “nodes” or “leaves” of the tree. 
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Placement recommendations were formed by setting certain criterion on the decision tree 
nodes, so that if students met or exceeded a minimum average probability of success at a 
certain level, they would be grouped together. The chosen criterion (70% predicted success 
rate at transfer level’) represents the minimum average successful completion rate of groups of 
students that the model places into that level of courses. 


In the decision trees, a machine-learning algorithm divides the data to create groups or “nodes” 
of past students with similar success rates based on the inputs (high school transcript 
information). For example, students with an HSGPA below 1.9 had a 43% predicted success rate 
in transfer-level English (the lowest node), while students with an HSGPA greater than or equal 
to 3.1 had an 87% predicted success rate (the highest node). The original goal of MMAP was to 
identify students who are highly likely to succeed at a particular course level in order to 
determine who should be placed directly into transfer-level courses. In contrast, Assembly Bill 
(AB) 705 created new guidelines requiring the identification of students who are highly unlikely 
to succeed if directly placed into transfer-level and determine whether their outcomes are 
better (i.e., “maximized”) if they start directly at transfer-level rather than in basic skills 
coursework. Thus, to help CCCs comply with AB 705, the MMAP research team focused on 
students in the lowest nodes (i.e., with the lowest high school performance) with the lowest 
likelihood of success, as these students are theoretically least likely to succeed in the target 
course and presumably most likely to benefit from placement into developmental education 
courses. While the student characteristics identified in these lowest nodes involve only HSGPA, 
the utility of other aspects of high school achievement are examined throughout but prove 
more important, for example, in higher nodes for math courses, such as whether or not a 
student took pre-calculus. 


Throughout this report, we examine the percentage of students who successfully complete 
transfer-level English or math courses with a grade of C or better within two primary semesters 
or three primary quarters of their first attempt in the discipline, also known as “throughput.” 
Importantly, throughput rates for students that start directly in a transfer-level course versus 
one level below are compared across three ranges of student high school performance as 
established by the original MMAP decision tree analysis. The top range is a combination of all 
nodes with success rates above 70%—students who are highly likely to succeed in the course as 
determined by the original MMAP research. The bottom range is the lowest terminal node, 
representing the lowest performing high school students who are least likely to succeed in the 
transfer-level course if placed there directly. The middle range comprises all of the nodes 
between the other two ranges of performance. Each of the three ranges for college 
composition (English), statistics, and pre-calculus are described in Table 1. 


8 The criterion was determined by the statewide Multiple Measures Work Group, a subcommittee of the Common 
Assessment Initiative, with discipline and counseling faculty as well as assessment and registration staff 
stakeholders from across the state represented. This criterion represents the approximate average statewide 
success rate in transfer-level English courses and is slightly higher than that rate in transfer-level mathematics 
courses. 
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Table 1. Decision Tree Nodes by Transfer-Level Subject 


Range 3 
Transfer-Level Range 1 Range 2 
: : p ll 2709 
Subject (lowest node) (all intermediate nodes) elses Ete 
success rate) 
English HS GPA<1.9 HS GPA2>1.9 and <2.6 HS GPA22.6 
Statistics? HS GPA<2.3 HS GPA2>2.3 and <3.0 HS GPA2>3.0 


HS GPA>3.4 
or 11th grade GPA>2.6 
with Calculus in HS 


HS GPA<2.6 andno HS GPA2>2.6 


Pre-calculus*® i : 
Pre-calculus in HS or Pre-calculus in HS 


Successful Completion of Transfer-Level 
English 
Summary of Throughput Rates for English by EOPS Status 


Comparing tables 2 and 3, EOPS students across all ranges of high school performance are one 
to two times more likely to complete the transfer-level course if they start at transfer-level than 
if they start one level below transfer. EOPS students in the lowest two ranges of high school 
performance are twice as likely to complete transfer-level English if they start there. 


Success Rates in Transfer-Level English for Students who Started at 
Transfer Level 


Table 2 displays success rates by EOPS status for students who started at transfer-level English. 
There were no differences in completion rates in the lowest range of high school performance, 
a 5 percentage point difference in the middle range, and a two percentage point difference in 
the highest range when looking at EOPS status. A larger proportion of EOPS students comprise 
the highest high school performance ranges than the lower ranges. 


Table 2. English Success Rates by EOPS Status—Transfer-Level 


Range 1 Range 2 Range 3 
EOPS HS GPA<1.9 HS GPA21.9 & <2.6 HS GPA22.6 
Rate N Rate N Rate N 
No 42% 6,644 59% 22355 80% 49,643 
Yes 42% 604 64% 1,580 78% 2397 


° Statistics includes a recommendation of as least successful completion of algebra I. 
10 Pre-calculus includes a recommendation of at least successful completion of Algebra II. 
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Throughput Rates for Students Entering One Level Below Transfer 


Table 3 displays throughput rates by EOPS status for students who started at one level below 
transfer-level English. Throughput rates were 8-13 percentage points higher for EOPS students 
than non-EOPS students across all ranges of high school performance. 


Table 3. English Throughput Rates by EOPS Status—One level Below 


Range 1 Range 2 Range 3 
EOPS HS GPA<1.9 HS GPA21.9 & <2.6 HS GPA22.6 
Rate N Rate N Rate N 
No 16% iil Ss; 29% 26,517 44% 29,425 
Yes 24% 1,249 39% 2,534 57% 2,367 


Summary of Throughput Rates for Transfer-Level English by 
DSPS Status 


DSPS students in the lowest range of high school performance are two times more likely to 
complete a transfer-level course if they start at transfer-level than if they start one level below 
transfer. Moreover, DSPS students in the middle range of high school performance are twice as 
likely to complete transfer-level English if they start there, and students in the highest range 
complete transfer-level English at a rate that is 25 percentage points higher than if placed one 
level below. 


Success Rates in Transfer-Level English for Students who Started at 
Transfer Level 


Table 4 displays success rates for students by DSPS status who started at transfer-level English. 
English success rates are similar when comparing DSPS and non-DSPS students; DSPS students 
complete transfer-level English courses at a rate that is just one percentage point higher in the 
lowest range of high school performance, one percentage point lower in the middle range, and 
at a rate three percentage points lower in the highest range. 


Table 4. English Success Rates by DSPS Status—Transfer-Level 


Range 1 Range 2 Range 3 
DSPS HS GPA<1.9 HS GPA21.9 & <2.6 HS GPA22.6 
Rate N Rate N Rate N 
No 42% 7,040 59% 23,345 80% 50,851 
Yes 43% 208 58% 590 77% 1,189 
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Throughput Rates Through Transfer-level English for Students Starting 
One-Level Below Transfer 


Table 5 displays the throughput rates by DSPS status for students who started at one level 
below transfer-level in English. English throughput rates are similar when comparing DSPS 
students to non-DSPS students in the two lowest ranges; DSPS students in the lowest range of 
high school achievement complete transfer-level English courses at the same rate as their non- 
DSPS peers, and DSPS students in the middle range complete transfer-level English at a rate 
that is two percentage point lower than their non-DSPS peers. However, DSPS students in the 
highest range of high school performance have English throughput rates that are eight 
percentage points greater when compared to non-DSPS students. 


Table 5. English Throughput Rates by DSPS Status—One level Below 


Range 1 Range 2 Range 3 
DSPS HS GPA<1.9 HS GPA21.9 & <2.6 HS GPA22.6 
Rate N Rate N Rate N 
No 17% 12,724 30% 28,071 44% 30,743 
Yes 17% 450 28% 980 52% 1,049 
Statistics 
EOPS 


Summary of Throughput Rates for Statistics by EOPS Status 


EOPS students in the lowest range of high school performance are nearly five times more likely 
to complete a transfer-level math course if they start in statistics than if they start one level 
below in mathematics. In addition, EOPS students in the higher ranges of high school 
performance are more than twice as likely to complete a transfer-level math course if they start 
in statistics. 


Success Rates in Statistics for Students who Started in Statistics 


Table 6 on the next page displays the success rates by EOPS status for students whose first 
math course was statistics. EOPS students exhibit higher success rates compared to non-EOPS 
students in the lowest range of high school performance, a five-percentage point difference. 


However, success rates were lower for EOPS students compared to non-EOPS students in both 
the middle and highest ranges of high school achievement, with a three and five percentage 
point difference, respectively. 


A smaller proportion of EOPS students were enrolled in statistics courses during the timeframe 
of this study compared to EOPS students enrolled in transfer-level English. 
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Table 6. Statistics Success Rates by EOPS Status 


Range 1 Range 2 Range 3 
EOPS HS GPA<2.3 HS GPA22.3 & <3.0 HS GPA23.0 
Rate N Rate N Rate N 
No 39% iLeisy? 57% 4,707 82% E350 
Yes 44% 133 54% 368 77% 365 


Throughput Rates to Statistics for Students Starting One-Level Below 
Statistics 

Table 7 displays throughput rates by EOPS status for students who started one level below any 
transfer-level math course. Throughput rates are similar for EOPS and non-EOPS students 


across all ranges of high school performance, with differences not exceeding three percentage 
points. 


Table 7. Statistics Throughput Rates by EOPS Status 


Range 1 Range 2 Range 3 
EOPS HS GPA<2.3 HS GPA22.3 & <3.0 HS GPA23.0 
Rate N Rate N Rate N 
No 9% 11O;225 18% 16,542 29% 9,647 
Yes 9% 1,084 21% 1,437 32% 739 


DSPS 


Summary of Throughput Rates for Statistics by DSPS Status 


Comparing tables 8 and 9, our analysis found that DSPS students in the middle and highest 
ranges exhibited throughput rates at least two times higher when placed directly into transfer- 
level statistics than if placed below any transfer-level math course. 


In the lowest range of high school performance, DSPS students exhibited throughput rates eight 
times higher when placed directly into transfer-level statistics than if placed below any transfer- 
level math course. 


Success Rates in Statistics for Students who Started in Statistics 


Table 8 on the following page displays the success rates by DSPS status for students whose first 
math course was statistics. DSPS students exhibited higher success rates in the lowest range by 
11 percentage points but lower success rates than non-DSPS students in the middle and highest 
ranges of high school performance, with success rates that were six and fourteen percentage 
points lower, respectively. 
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Table 8. Statistics Success Rates by DSPS Status 


Range 1 Range 2 Range 3 
DSPS HS GPA<2.3 HS GPA22.3 & <3.0 HS GPA23.0 
Rate N Rate N Rate N 
No 39% 1,451 57% 4,976 82% Tes 
Yes 50% 34 51% 99 68% 142 


Throughput Rates to Transfer-Level for Students Starting One-Level 
Below Transfer 

Table 9 displays throughput rates by DSPS status for students who started one level below 
transfer-level math. There was no more than a three-percentage point difference between the 


throughput rates of DSPS and non-DSPS students, with DSPS students having slightly higher 
throughput rates in the two highest ranges of high school performance. 


Table 9. One level Below Transfer-Level Throughput Rates by DSPS Status 


Range 1 Range 2 Range 3 
DSPS HS GPA<2.3 HS GPA22.3 & <3.0 HS GPA23.0 
Rate N Rate N Rate N 
No 9% 11,004 18% 17,504 29% OM27 
Yes 6% 305 21% 475 30% 259 


Pre-Calculus 


Summary of Throughput Rates for Pre-Calculus by EOPS Status 


EOPS students in the lowest high school performance range who start in pre-calculus are almost 
three times as likely to complete that course than students who begin the business-STEM 
(BSTEM) math sequence at one level below transfer are to complete any transfer-level math 
class. 


Additionally, across all ranges of high school performance, throughput rates improved by 28 
percentage points when students were placed directly into pre-calculus compared to starting 
one level below any transfer-level math course. 


EOPS 


Success Rates to Pre-calculus for Students who Started in Pre-Calculus 


Table 10 on the next page displays success rates by EOPS status for students whose first math 
course was pre-calculus. Success rates for EOPS students were four percentage points higher 
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than for non-EOPS students in the lowest range of high school performance. In the middle and 
highest ranges of high school performance, success rates for EOPS students and non-EOPS 
students differed by only one percentage point with EOPS students performing slightly higher in 
the middle range and slightly lower in the highest range. 


Table 10. Pre-Calculus Success Rates by EOPS Status 


Range 1 Range 2 Range 3 
> 

HSGPAG Cando HSGPASD GORE? OS Ce ee 
EOPS A : grade GPA22.6 with 

Pre-calculus in HS calculus in HS z 

Calculus in HS 
Rate N Rate N Rate N 

No 38% 1,587 58% 4,458 78% 1,976 
Yes 42% 166 59% 323 77% 138 


Throughput Rates to Pre-calculus for Students Starting One-Level Below 
Transfer 


Table 11 displays throughput rates to completion of any business-STEM transfer-level math 
course by EOPS status for students whose first course was one level below transfer-level. 
Throughput rates for EOPS students were one to two percentage points higher than those of 
non-EOPS students in the lowest and highest ranges of high school performance. In the middle 
range of high school performance, EOPS students had throughout rates 2 percentage points 
lower than those of non-EOPS students. 


Table 11. Pre-Calculus Throughput Rates by EOPS Status 


Range 1 Range 2 Range 3 
HS GPA23.4 or 11th 


HS GPA<2.6 and no HS GPA22.6 or Pre- 


EOFS Pre-calculus in HS calculus in HS grade pean 2” 
Calculus in HS 
Rate N Rate N Rate N 
No 14% 17,265 32% 16,867 47% 3,281 
Yes 15% 1,652 30% 1351 49% 257 


DSPS 


Summary of Throughput Rates for Pre-calculus by DSPS Status 


DSPS students in the lowest high school performance range who are placed directly into pre- 
calculus have throughput rates that are 33 percentage points higher than those of students 
who start one level below transfer-level in mathematics. In the middle high school performance 
range, DSPS students who are placed directly into pre-calculus have throughput rates that are 
13 percentage points higher than those of students who start one level below transfer-level in 
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mathematics. Students in the highest high school performance band have throughput rates that 
are 18 percentage points higher than students who start one level below transfer-level in 
mathematics. 


The highest performing DSPS students resulted in a small sample size (77 students), and are 
more likely to complete any BSTEM transfer-level math if they start one-level below than if they 
start in pre-calculus. However, given the very small sample size, the difference fails to approach 
significance. Further, the adjustments applied to the samples, particularly for BSTEM work well 
for large sample sizes, but are less appropriate for small samples. 


Success Rates in Pre-calculus for Students who Started in Pre- 
Calculus 


Table 12 displays success rates by DSPS status for students whose first math course was pre- 
calculus. The success rate for DSPS students in the lowest high school performance range was 
eight percentage points higher than non-DSPS students in the same range. However, DSPS 
students in the middle and highest ranges had lower success rates in precalculus than non-DSPS 
students, with differences in success rates ranging from eight to ten percentage points lower 
for DSPS students. 


Table 12. Pre-Calculus Success Rates by DSPS Status 


Range 1 Range 2 Range 3 


HS GPA23.4 or 11th 
HS GPA<2.6 and no HS GPA22.6 or Pre- ‘ 
DSPS : : grade GPA22.6 with 
Pre-calculus in HS calculus in HS ‘ 
Calculus in HS 


Rate N Rate N Rate N 
No 38% 1,719 58% 4,689 78% 2,064 
Yes 46% 34 48% 92 70% 50 


Throughput Rates to Business-STEM Transfer-Level Course for Students 
Starting One Level Below Transfer 


Table 13 on the next page displays throughput rates through any business-STEM transfer-level 
math course by DSPS status for students whose first course was one level below transfer. There 
was almost no difference in the success rates of DSPS and non-DSPS students in the lowest 
range of high school performance. However, DSPS students had higher throughput rates than 
non-DSPS students in the middle and highest ranges of high school performance, a 2 to 18 
percentage point difference, respectively. 
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Table 13. One Level Below Business-STEM Transfer-Level Courses Throughput 
Rates by DSPS Status 


Range 1 Range 2 Range 3 


HS GPA23.4 or 11th 
HS GPA<2.6 and no HS GPA22.6 or Pre- eG 
DSPS ; : grade GPA22.6 with 
Pre-calculus in HS calculus in HS : 
Calculus in HS 


Rate N Rate N Rate N 
No 14% 18,414 32% 17,759 46% 3,461 
Yes 13% 503 35% 459 88% Gd 


Conclusion 


For all three gateway courses, all student groups had higher levels of success at the transfer 
level when they were placed directly into those transfer-level courses rather than beginning at 
a lower level. Throughput rates were one to five times higher among students whose first 
course was transfer-level English, statistics, or pre-calculus—an advantage that held across 
EOPS/DSPS status as well as all ranges of high school performance. Moreover, the benefit of 
direct enrollment at the transfer level was strongest for students in the lowest range of high 
school performance. 


Within the timeframe of data availability and given the curricular design and support structures 
that existed systemwide at this time, virtually every student group”! is more likely to 
successfully complete transfer-level courses when placed directly into the transfer-level 
course in all subjects examined. 


It is recommended that each college conduct its own analysis to compare throughput rates for 
students who begin below transfer-level to success rates among those who start at transfer- 
level at each level of high school achievement, disaggregated by special population status, both 
with and without specialized support, such as co-requisites, to ensure that local data align with 
the statewide findings. Further, colleges are encouraged to evaluate and assess their placement 
process, curricular design, concurrent supports and non-curricular support as well as determine 
and address disproportionate outcomes for historically underrepresented populations. 


11 With only a few minor exceptions of low sample size. The types of adjustments used in this report are not well 
suited for small sample sizes. The BSTEM adjustment in particular works for large populations but is less 
appropriate for some sub-populations. Further, the success rates for these groups if placed directly into pre- 
calculus (compared to their throughput to any BSTEM course) very strongly suggests that they are not at all close 
to highly unlikely to succeed and so still should not be placed one-level below transfer-level. 
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